Toward Learning Mixture-of-Parts Pictorial Structures

For many multi-part visual object classes, the set of parts can vary not only in location but also in type. For example, player formations in American football involve various subsets of player types, and the spatial constraints between players depend largely upon which subset of player types constitutes the formation. In this paper, we consider the problem of learning to jointly localize and classify the parts of such objects, driven by our application focus in the domain of American football. Standard models from computer vision and structured machine learning do not appear adequate for our problem class, and we have in turn developed the mixture-of-parts pictorial structure (MoPPS) model which allows for joint constraints on the types and locations of object parts. Here we review the MoPPS model and its application in the football domain, and we discuss opportunities for learning suggested by our experience, including opportunities for structure and parameter learning, speed-up learning, active learning, and transfer learning.